Message Passing Algorithms for Dirichlet Diffusion Trees
نویسندگان
چکیده
We demonstrate efficient approximate inference for the Dirichlet Diffusion Tree (Neal, 2003), a Bayesian nonparametric prior over tree structures. Although DDTs provide a powerful and elegant approach for modeling hierarchies they haven’t seen much use to date. One problem is the computational cost of MCMC inference. We provide the first deterministic approximate inference methods for DDT models and show excellent performance compared to the MCMC alternative. We present message passing algorithms to approximate the Bayesian model evidence for a specific tree. This is used to drive sequential tree building and greedy search to find optimal tree structures, corresponding to hierarchical clusterings of the data. We demonstrate appropriate observation models for continuous and binary data. The empirical performance of our method is very close to the computationally expensive MCMC alternative on a density estimation problem, and significantly outperforms kernel density estimators.
منابع مشابه
A Unifying View of Message Passing algorithms for Gaussian MRFs
We illustrate the relationship between message passing algorithms in GMRFs and matrix factorization algorithms. Specifically, we show that message passing on trees is equivalent to Gaussian elimination, while Loopy Belief Propagation is equivalent to Gauss-Seidel relaxation. Similarly, recently introduced message passing algorithms such as the Extended Message Passing, and the Embedded Subgraph...
متن کاملAn Operator Interpretation of Message Passing
Message passing algorithms may be viewed from a purely probabilistic or statistical physics perspective. This works describes an alternative, linear algebraic view of message passing in trees. We demonstrate the construction of global belief operators on Markov Chains and Trees and compare these with classical results. By interpreting message passing as finding a global stable solution we wish ...
متن کاملMemory-Efficient Topic Modeling
As one of the simplest probabilistic topic modeling techniques, latent Dirichlet allocation (LDA) has found many important applications in text mining, computer vision and computational biology. Recent training algorithms for LDA can be interpreted within a unified message passing framework. However, message passing requires storing previous messages with a large amount of memory space, increas...
متن کاملHybrid Parallel Inference for Hierarchical Dirichlet Processes
The hierarchical Dirichlet process (HDP) can provide a nonparametric prior for a mixture model with grouped data, where mixture components are shared across groups. However, the computational cost is generally very high in terms of both time and space complexity. Therefore, developing a method for fast inference of HDP remains a challenge. In this paper, we assume a symmetric multiprocessing (S...
متن کامل